Periodic generation of intermediate summaries to facilitate report acceleration

ABSTRACT

A method and system for managing searches of a data set that is partitioned based on a plurality of events. A structure of a search query may be analyzed to determine if logical computational actions performed on the data set is reducible. Data in each partition is analyzed to determine if at least a portion of the data in the partition is reducible. In response to a subsequent or reoccurring search request, intermediate summaries of reducible data and reducible search computations may be aggregated for each partition. Next, a search result may be generated based on at least one of the aggregated intermediate summaries, the aggregated reducible search computations, and a query of adhoc non-reducible data arranged in at least one of the plurality of partitions for the data set.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims benefit under 35 U.S.C. § 120 as a Continuationof Ser. No. 14/530,678, filed Oct. 31, 2014, which claims benefit as aContinuation of U.S. application Ser. No. 14/168,738, filed on Jan. 30,2014, now issued as U.S. Pat. No. 9,177,002, which claims benefit as aContinuation of U.S. application Ser. No. 13/664,239, filed on Oct. 30,2012, now issued as U.S. Pat. No. 8,682,886, which claims the benefit ofU.S. Provisional Application 61/649,125 filed on May 18, 2012. Theentire contents of each of the above-listed applications is herebyincorporated by reference for all purposes as if fully set forth herein.

TECHNICAL FIELD

The invention is directed to providing services for use by a customer,and more particularly, to providing search services to the customer overa network.

BACKGROUND OF THE INVENTION

Many companies install and maintain search software for use on a varietyof distributed systems that can range from a single computer for a smallbusiness to a collection of servers and a plurality of user computernodes for a large corporation. In the past, search reports based onrelatively large sets of indexed data were time consuming to generate.To reduce latency in search report generation for a larger data set,intermediate summaries for reoccurring search reports for partitioneddata have been periodically generated and stored in a separate index.Subsequently, a search report run on the data set would aggregate thecorresponding pre-computed intermediate summaries to generate the searchreport in a relatively shorter period of time. However, the nature ofdifferent types of data and the structure of different search querieshas made it difficult to manage and configure these intermediatesummaries. For example, some partitions may include data that is highlyreducible for a search query and other partitions may include data thatis only marginally reducible for the same search query. Also, for asingle set of data in a partition, it may be reducible for some queriesbut not others. Other difficulties include configuring the amount ofdata included in the partitions of the data set, such as adding newlyidentified data to a partition after intermediate summaries aregenerated. Consequently, systems that can manage search reportgeneration for relatively large sets of data are the subject ofconsiderable innovation.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments are described with referenceto the following drawings. In the drawings, like reference numeralsrefer to like parts throughout the various figures unless otherwisespecified.

For a better understanding, reference will be made to the followingDescription Of The Various Embodiments, which is to be read inassociation with the accompanying drawings, wherein:

FIG. 1 illustrates a system environment in which various embodiments maybe implemented;

FIG. 2A shows a schematic drawing of a rack of blade servers;

FIG. 2B illustrates a schematic embodiment of a blade server that may beincluded in a rack of blade servers such as that shown in FIG. 2A;

FIG. 3 shows a schematic embodiment of a mobile device;

FIG. 4 illustrates a schematic embodiment of a network device;

FIG. 5 shows a flowchart for general processing a data set and provide aresult in response to a search query;

FIG. 6 illustrates a flowchart for processing intermediate summaries andreducible search computations for partitioned data;

FIG. 7 shows a flowchart for processing a search request;

FIG. 8 illustrates a schematic diagram of a data set that includes aplurality of partitions;

FIG. 9 shows an exemplary table of raw data; and

FIG. 10 illustrates an exemplary table of intermediate summaries inaccordance with the various embodiments.

DESCRIPTION OF THE VARIOUS EMBODIMENTS

The invention now will be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific embodiments by which theinvention may be practiced. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. Amongother things, the invention may be embodied as methods or devices.Accordingly, the invention may take the form of an entirely hardwareembodiment, an entirely software embodiment or an embodiment combiningsoftware and hardware aspects. The following detailed description is,therefore, not to be taken in a limiting sense.

Throughout the specification and claims, the following terms take themeanings explicitly associated herein, unless the context clearlydictates otherwise. The phrase “in one embodiment” as used herein doesnot necessarily refer to the same embodiment, though it may.Furthermore, the phrase “in another embodiment” as used herein does notnecessarily refer to a different embodiment, although it may. Thus, asdescribed below, various embodiments may be readily combined, withoutdeparting from the scope or spirit of the invention.

In addition, as used herein, the term “or” is an inclusive “or”operator, and is equivalent to the term “and/or,” unless the contextclearly dictates otherwise. The term “based on” is not exclusive andallows for being based on additional factors not described, unless thecontext clearly dictates otherwise. In addition, throughout thespecification, the meaning of “a,” “an,” and “the” include pluralreferences. The meaning of “in” includes “in” and “on.”

Various embodiments now will be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific exemplary embodiments bywhich the invention may be practiced. The embodiments may, however, beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the embodiments to those skilled in the art.Among other things, the various embodiments may be methods, systems,media or devices. Accordingly, the various embodiments may take the formof an entirely hardware embodiment, an entirely software embodiment oran embodiment combining software and hardware aspects. The followingdetailed description is, therefore, not to be taken in a limiting sense.

For example embodiments, the following terms are also used hereinaccording to the corresponding meaning, unless the context clearlydictates otherwise.

The term “query” as used herein refers to commands and/or sequences ofcommands that are directed for searching and/or retrieving data from arecord datastore. Queries generally produce a result or results based onthe form and structure of the particular query. Query results may besorted and grouped based on the structure and form of the query. In atleast one of the various embodiments, queries may includes operators andfunctions for calculating value based on the stored records, includingfunctions that produce results sets that may include statistics andmetrics about the portions of the record datastore. Structure QueryLanguage (SQL) is a well-know query language often used to form queriesfor relational databases; herein queries may be described using a“SQL-like’ form for readability. However, the various embodiments arenot limited to using SQL-like formatting for queries and other well knowquery languages and/or custom query languages may be employed consistentwith what is claimed herein.

“Index”, “Indexer”, “Indexing”, and “Index Storage” as used herein mayrepresent elements of described embodiments that may index and storedata and events. Indexers may collect, parse, and store data tofacilitate fast and accurate information retrieval. Index design mayincorporate interdisciplinary concepts from linguistics, cognitivepsychology, mathematics, informatics, physics, and computer science.Also, Indexes may reside in flat files in a data store on a file systemwhich may be distributed and enables distributed computations. Indexfiles may be managed to facilitate flexible searching and fast dataretrieval, eventually archiving them according to a configurableschedule. During indexing, incoming raw data may be processed to enablefast search and analysis, the results of which may be stored in anindex. As part of the indexing process, the indexer may add knowledge tothe data in various ways, including by: separating a data stream intoindividual, searchable events; creating or identifying timestamps;extracting fields such as host, source, and source type; performinguser-defined actions on the incoming data, such as identifying customfields, masking sensitive data, writing new or modified keys, applyingbreaking rules for multi-line events, filtering unwanted events, routingevents to specified indexes or servers, and the like. Also, for areducible structure of a reoccurring search, the indexer can perform alogical sequence of actions on the data, e.g., computing averages,variances, root mean squares, or the like.

Briefly stated, various embodiments are directed towards a system andmethod for managing and configuring one or more different search reportsfor a data set that is partitioned based on a time of occurrence for aplurality of events. Data corresponding to each event may be arranged ina corresponding partition. In at least one of the various embodiments, astructure of a search query for a particular search report may beanalyzed to determine if at least a portion of the query's logicalcomputational actions are reducible. If so, the reducible searchcomputations are performed on at least one of the partitions. Also, inat least one of the various embodiments, data in each partition isanalyzed to determine if at least a portion of the data is reducible fora particular search report. If so, intermediate summaries are generatedfor the reducible data which may be arranged in one or more partitions.In response to a subsequent or reoccurring request for a particularsearch report, the corresponding intermediate summaries and reduciblesearch computations may be aggregated for each partition. Next, a searchresult may be generated based on at least one of the aggregatedintermediate summaries, the aggregated reducible search computations,and a search query of any remaining adhoc non-reducible data arranged inat least one of the plurality of partitions for the data set.Additionally, intermediate summaries and reducible search computationsmay be reused for more than one particular search report.

In at least one of the various embodiments, each partition of the dataset is based on at least a time for an event. Also, in at least one ofthe various embodiments, the data may include time series data whereeach data record has a timestamp associated with it. Search reports forthe stored time series data may be limited by a range of time (such asbetween one time and another, or data earlier than a given time, and thelike) and/or provide search results ordered by a time (such as fromearliest-to-latest or vice versa). Additionally, in at least one of thevarious embodiments, newly identified other data related to an event maybe appended to a particular partition that corresponds to the event.

In at least one of the various embodiments, at least a portion of atleast one partition of the data set can include adhoc data that issubstantially unique/non-reducible for a particular search report. Forexample, in at least one of the various embodiments, if the amount ofadhoc non-reducible data in a partition for a particular search reportis substantially greater than reducible data, the process may stopgenerating further intermediate summaries for that partition for thatparticular search report unless more reducible data is appended to thepartition. In at least one of the various embodiments, reoccurrences ofthe search report may be one of periodic, intermittent, or random.

In at least one of the various embodiments, a logical sequence ofactions performed by a reducible search computation for a particularsearch report may be normalized for the partitioned data set.Additionally, in at least one of the various embodiments, a uniquenessvalue for at least a portion of the data in each partition for aparticular search report may be determined. Also, a high uniquenessvalue may be employed to identify the non-reducible data and a lowuniqueness value may be employed to identify reducible data in eachpartition for a particular search report. Additionally, the data may bestored in one or more data stores, and one or more computing devices maybe employed to distribute the computing of intermediate summaries,reducible search computations, and search results for a plurality ofdifferent search reports.

The following briefly describes embodiments in order to provide a basicunderstanding of some aspects of the invention. This brief descriptionis not intended as an extensive overview. It is not intended to identifykey or critical elements, or to delineate or otherwise narrow the scope.Its purpose is merely to present some concepts in a simplified form as aprelude to the more detailed description that is presented later.

Additionally, it should be appreciated that though many embodimentsdescribed herein may be cloud-based, embodiments should not beconsidered so limited. One of ordinary skill in the art will recognizethat enabling embodiments may be arranged to be partially deployed incloud-based configurations with some elements in the cloud and someelements separate from cloud based resources. Likewise, enablingembodiments may be arranged to be deployed and operate in configurationsentirely separate from cloud-based resources.

Illustrative Operating Environment

FIG. 1 shows components of an environment in which various embodimentsmay be practiced. Not all of the components may be required to practicethe various embodiments, and variations in the arrangement and type ofthe components may be made without departing from the spirit or scope ofthe various embodiments.

In at least one embodiment, cloud network 102 enables one or morenetwork services for a user based on the operation of correspondingarrangements 104 and 106 of virtually any type of networked computingdevice. As shown, the networked computing devices may include servernetwork device 112, host network device 114, enclosure of blade servers110, enclosure of server computers 116, super computer network device118, and the like. Although not shown, one or more mobile devices may beincluded in cloud network 102 in one or more arrangements to provide oneor more network services to a user. Also, these arrangements ofnetworked computing devices may or may not be mutually exclusive of eachother.

Additionally, the user may employ a plurality of virtually any type ofwired or wireless networked computing devices to communicate with cloudnetwork 102 and access at least one of the network services enabled byone or more of arrangements 104 and 106. These networked computingdevices may include tablet mobile device 122, handheld mobile device124, wearable mobile device 126, desktop network device 120, and thelike. Although not shown, in various embodiments, the user may alsoemploy notebook computers, desktop computers, microprocessor-based orprogrammable consumer electronics, network appliances, mobiletelephones, smart telephones, pagers, radio frequency (RF) devices,infrared (IR) devices, Personal Digital Assistants (PDAs), televisions,integrated devices combining at least one of the preceding devices, andthe like.

One embodiment of a mobile device is described in more detail below inconjunction with FIG. 3. Generally, mobile devices may include virtuallyany substantially portable networked computing device capable ofcommunicating over a wired, wireless, or some combination of wired andwireless network.

In various embodiments, network 102 may employ virtually any form ofcommunication technology and topology. For example, network 102 caninclude local area networks Personal Area Networks (PANs), (LANs),Campus Area Networks (CANs), Metropolitan Area Networks (MANs) Wide AreaNetworks (WANs), direct communication connections, and the like, or anycombination thereof. On an interconnected set of LANs, including thosebased on differing architectures and protocols, a router acts as a linkbetween LANs, enabling messages to be sent from one to another. Inaddition, communication links within networks may include virtually anytype of link, e.g., twisted wire pair lines, optical fibers, open airlasers or coaxial cable, plain old telephone service (POTS), waveguides, acoustic, full or fractional dedicated digital communicationlines including T1, T2, T3, and T4, and/or other carrier and other wiredmedia and wireless media. These carrier mechanisms may includeE-carriers, Integrated Services Digital Networks (ISDNs), universalserial bus (USB) ports, Firewire ports, Thunderbolt ports, DigitalSubscriber Lines (DSLs), wireless links including satellite links, orother communications links known to those skilled in the art. Moreover,these communication links may further employ any of a variety of digitalsignaling technologies, including without limit, for example, DS-0,DS-1, DS-2, DS-3, DS-4, OC-3, OC-12, OC-48, or the like. Furthermore,remotely located computing devices could be remotely connected tonetworks via a modem and a temporary communication link. In essence,network 102 may include virtually any communication technology by whichinformation may travel between computing devices. Additionally, in thevarious embodiments, the communicated information may include virtuallyany kind of information including, but not limited to processor-readableinstructions, data structures, program modules, applications, raw data,control data, archived data, video data, voice data, image data, textdata, and the like.

Network 102 may be partially or entirely embodied by one or morewireless networks. A wireless network may include any of a variety ofwireless sub-networks that may further overlay stand-alone ad-hocnetworks, and the like. Such sub-networks may include mesh networks,Wireless LAN (WLAN) networks, Wireless Router (WR) mesh, cellularnetworks, pico networks, PANs, Open Air Laser networks, Microwavenetworks, and the like. Network 102 may further include an autonomoussystem of intermediate network devices such as terminals, gateways,routers, switches, firewalls, load balancers, and the like, which arecoupled to wired and/or wireless communication links. These autonomousdevices may be operable to move freely and randomly and organizethemselves arbitrarily, such that the topology of network 102 may changerapidly.

Network 102 may further employ a plurality of wired and wireless accesstechnologies, e.g., 2nd (2G), 3rd (3G), 4th (4G), 5^(th) (5G) generationwireless access technologies, and the like, for mobile devices. Thesewired and wireless access technologies may also include Global Systemfor Mobile communication (GSM), General Packet Radio Services (GPRS),Enhanced Data GSM Environment (EDGE), Code Division Multiple Access(CDMA), Wideband Code Division Multiple Access (WCDMA), Long TermEvolution Advanced (LTE), Universal Mobile Telecommunications System(UMTS), Orthogonal frequency-division multiplexing (OFDM), Wideband CodeDivision Multiple Access (W-CDMA), Code Division Multiple Access 2000(CDMA2000), Evolution-Data Optimized (EV-DO), High-Speed Downlink PacketAccess (HSDPA), IEEE 802.16 Worldwide Interoperability for MicrowaveAccess (WiMax), ultra wide band (UWB), user datagram protocol (UDP),transmission control protocol/Internet protocol (TCP/IP), any portion ofthe Open Systems Interconnection (OSI) model protocols, Short MessageService (SMS), Multimedia Messaging Service (MMS), Web Access Protocol(WAP), Session Initiation Protocol/Real-time Transport Protocol(SIP/RTP), or any of a variety of other wireless or wired communicationprotocols. In one non-limiting example, network 102 may enable a mobiledevice to wirelessly access a network service through a combination ofseveral radio network access technologies such as GSM, EDGE, SMS, HSDPA,LTE and the like.

Enclosure of Blade Servers

FIG. 2A shows one embodiment of an enclosure of blade servers 200, whichare also illustrated in FIG. 1. Enclosure of blade servers 200 mayinclude many more or fewer components than those shown in FIG. 2A.However, the components shown are sufficient to disclose an illustrativeembodiment. Generally, a blade server is a stripped down servercomputing device with a modular design optimized to minimize the use ofphysical space and energy. A blade enclosure can include several bladeservers and provide each with power, cooling, network interfaces,input/output interfaces, and resource management. Although not shown, anenclosure of server computers typically includes several computers thatmerely require a network connection and a power cord connection tooperate. Each server computer often includes redundant components forpower and interfaces.

As shown in the figure, enclosure 200 contains power supply 204, andinput/output interface 206, rack logic 208, several blade servers 210,212, 214, and 216, and backplane 202. Power supply 204 provides power toeach component and blade server within the enclosure. The input/outputinterface 206 provides internal and external communication forcomponents and blade servers within the enclosure. Backplane 208 canenable passive and active communication of power, logic, input signals,and output signals for each blade server.

Illustrative Blade Server

FIG. 2B illustrates an illustrative embodiment of blade server 250,which may include many more or fewer components than those shown. Asshown in FIG. 2A, a plurality of blade servers may be included in oneenclosure that shares resources provided by the enclosure to reducesize, power, and cost.

Blade server 250 includes processor 252 which communicates with memory256 via bus 254. Blade server 250 also includes input/output interface290, processor-readable stationary storage device 292, andprocessor-readable removable storage device 294. Input/output interface290 can enable blade server 250 to communicate with other blade servers,mobile devices, network devices, and the like. Interface 190 may providewireless and/or wired communication links for blade server.Processor-readable stationary storage device 292 may include devicessuch as an electromagnetic storage device (hard disk), solid state harddisk (SSD), hybrid of both an SSD and a hard disk, and the like. Also,processor-readable removable storage device 294 enables processor 252 toread non-transitive storage media for storing and accessingprocessor-readable instructions, modules, data structures, and otherforms of data. The non-transitive storage media may include Flashdrives, tape media, floppy media, and the like.

Memory 256 may include Random Access Memory (RAM), Read-Only Memory(ROM), hybrid of RAM and ROM, and the like. As shown, memory 256includes operating system 258 and basic input/output system (BIOS) 260for enabling the operation of blade server 250. In various embodiments,a general-purpose operating system may be employed such as a version ofUNIX, or LINUX™, or a specialized server operating system such asMicrosoft's Windows Server™ and Apple Computer's IoS Server™.

Memory 256 further includes one or more data storage 270, which can beutilized by blade server 250 to store, among other things, applications280 and/or other data. Data stores 270 may include program code, data,algorithms, and the like, for use by processor 252 to execute andperform actions. In one embodiment, at least some of data store 270might also be stored on another component of blade server 250,including, but not limited to, processor-readable removable storagedevice 294, processor-readable stationary storage device 292, or anyother processor-readable storage device (not shown). Data storage 270may include, for example, raw data 272, indexed data 274, andintermediate summaries 276.

Applications 280 may include processor executable instructions which,when executed by blade server 250, transmit, receive, and/or otherwiseprocess messages, audio, video, and enable communication with othernetworked computing devices. Examples of application programs includedatabase servers, file servers, calendars, transcoders, and so forth.Applications 280 may include, for example, search application 282, andindexing application 284.

Human interface components (not pictured), may be remotely associatedwith blade server 250, which can enable remote input to and/or outputfrom blade server 250. For example, information to a display or from akeyboard can be routed through the input/output interface 290 toappropriate peripheral human interface components that are remotelylocated. Examples of peripheral human interface components include, butare not limited to, an audio interface, a display, keypad, pointingdevice, touch interface, and the like.

Illustrative Mobile Device

FIG. 3 shows one embodiment of mobile device 300 that may include manymore or less components than those shown. Mobile device 300 mayrepresent, for example, at least one embodiment of mobile devices shownin FIG. 1.

Mobile device 300 includes processor 302 in communication with memory304 via bus 328. Mobile device 300 also includes power supply 330,network interface 332, audio interface 356, display 350, keypad 352,illuminator 354, video interface 342, input/output interface 338, hapticinterface 364, global positioning systems (GPS) receiver 358, Open airgesture interface 360, temperature interface 362, camera(s) 340,projector 346, pointing device interface 366, processor-readablestationary storage device 334, and processor-readable removable storagedevice 336. Power supply 330 provides power to mobile device 300. Arechargeable or non-rechargeable battery may be used to provide power.The power may also be provided by an external power source, such as anAC adapter or a powered docking cradle that supplements and/or rechargesthe battery. And in one embodiment, although not shown, a gyroscope maybe employed within mobile device 300 to measuring and/or maintaining anorientation of mobile device 300.

Mobile device 300 may optionally communicate with a base station (notshown), or directly with another computing device. Network interface 332includes circuitry for coupling mobile device 300 to one or morenetworks, and is constructed for use with one or more communicationprotocols and technologies including, but not limited to, protocols andtechnologies that implement any portion of the Open SystemsInterconnection (OSI) model for mobile communication (GSM), codedivision multiple access (CDMA), time division multiple access (TDMA),user datagram protocol (UDP), transmission control protocol/Internetprotocol (TCP/IP), Short Message Service (SMS), Multimedia MessagingService (MMS), general packet radio service (GPRS), Web Access Protocol(WAP), ultra wide band (UWB), IEEE 802.16 Worldwide Interoperability forMicrowave Access (WiMax), Session Initiation Protocol/Real-timeTransport Protocol (SIP/RTP), General Packet Radio Services (GPRS),Enhanced Data GSM Environment (EDGE), Wideband Code Division MultipleAccess (WCDMA), Long Term Evolution Advanced (LTE), Universal MobileTelecommunications System (UMTS), Orthogonal frequency-divisionmultiplexing (OFDM), Code Division Multiple Access 2000 (CDMA2000),Evolution-Data Optimized (EV-DO), High-Speed Downlink Packet Access(HSDPA), or any of a variety of other wireless communication protocols.Network interface 332 is sometimes known as a transceiver, transceivingdevice, or network interface card (NIC).

Audio interface 356 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 356 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others and/or generate an audio acknowledgementfor some action. A microphone in audio interface 356 can also be usedfor input to or control of mobile device 300, e.g., using voicerecognition, detecting touch based on sound, and the like.

Display 350 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computing device. Display 350 may also include a touchinterface 344 arranged to receive input from an object such as a stylusor a digit from a human hand, and may use resistive, capacitive, surfaceacoustic wave (SAW), infrared, radar, or other technologies to sensetouch and/or gestures. Projector 346 may be a remote handheld projectoror an integrated projector that is capable of projecting an image on aremote wall or any other reflective object such as a remote screen.

Video interface 342 may be arranged to capture video images, such as astill photo, a video segment, an infrared video, or the like. Forexample, video interface 342 may be coupled to a digital video camera, aweb-camera, or the like. Video interface 342 may comprise a lens, animage sensor, and other electronics. Image sensors may include acomplementary metal-oxide-semiconductor (CMOS) integrated circuit,charge-coupled device (CCD), or any other integrated circuit for sensinglight.

Keypad 352 may comprise any input device arranged to receive input froma user. For example, keypad 352 may include a push button numeric dial,or a keyboard. Keypad 352 may also include command buttons that areassociated with selecting and sending images. Illuminator 354 mayprovide a status indication and/or provide light. Illuminator 354 mayremain active for specific periods of time or in response to events. Forexample, when illuminator 354 is active, it may backlight the buttons onkeypad 352 and stay on while the mobile device is powered. Also,illuminator 354 may backlight these buttons in various patterns whenparticular actions are performed, such as dialing another mobile device.Illuminator 354 may also cause light sources positioned within atransparent or translucent case of the mobile device to illuminate inresponse to actions.

Mobile device 300 also comprises input/output interface 338 forcommunicating with external peripheral devices or other computingdevices such as other mobile devices and network devices. The peripheraldevices may include an audio headset, display screen glasses, remotespeaker system, remote speaker and microphone system, and the like.Input/output interface 338 can utilize one or more technologies, such asUniversal Serial Bus (USB), Infrared, WiFi, WiMax, Bluetooth™, and thelike. Haptic interface 364 is arranged to provide tactile feedback to auser of the mobile device. For example, the haptic interface 364 may beemployed to vibrate mobile device 300 in a particular way when anotheruser of a computing device is calling. Temperature interface 362 may beused to provide a temperature measurement input and/or a temperaturechanging output to a user of mobile device 300. Open air gestureinterface 360 may sense physical gestures of a user of mobile device300, for example, by using single or stereo video cameras, radar, agyroscopic sensor inside a device held or worn by the user, or the like.Camera 340 may be used to track physical eye movements of a user ofmobile device 300.

GPS transceiver 358 can determine the physical coordinates of mobiledevice 300 on the surface of the Earth, which typically outputs alocation as latitude and longitude values. GPS transceiver 358 can alsoemploy other geo-positioning mechanisms, including, but not limited to,triangulation, assisted GPS (AGPS), Enhanced Observed Time Difference(E-OTD), Cell Identifier (CI), Service Area Identifier (SAI), EnhancedTiming Advance (ETA), Base Station Subsystem (BSS), or the like, tofurther determine the physical location of mobile device 300 on thesurface of the Earth. It is understood that under different conditions,GPS transceiver 358 can determine a physical location for mobile device300. In at least one embodiment, however, mobile device 300 may, throughother components, provide other information that may be employed todetermine a physical location of the device, including for example, aMedia Access Control (MAC) address, IP address, and the like.

Human interface components can be peripheral devices that are physicallyseparate from mobile device 300, allowing for remote input and/or outputto mobile device 300. For example, information routed as described herethrough human interface components such as display 350 or keyboard 352can instead be routed through network interface 332 to appropriate humaninterface components located remotely. Examples of human interfaceperipheral components that may be remote include, but are not limitedto, audio devices, pointing devices, keypads, displays, cameras,projectors, and the like. These peripheral components may communicateover a Pico Network such as Bluetooth™, Zigbee™ and the like. Onenon-limiting example of a mobile device with such peripheral humaninterface components is a wearable computing device, which might includea remote pico projector along with one or more cameras that remotelycommunicate with a separately located mobile device to sense a user'sgestures toward portions of an image projected by the pico projectoronto a reflected surface such as a wall or the user's hand.

A mobile device may include a browser application that is configured toreceive and to send web pages, web-based messages, graphics, text,multimedia, and the like. The mobile device's browser application mayemploy virtually any programming language, including a wirelessapplication protocol messages (WAP), and the like. In at least oneembodiment, the browser application is enabled to employ Handheld DeviceMarkup Language (HDML), Wireless Markup Language (WML), WMLScript,JavaScript, Standard Generalized Markup Language (SGML), HyperTextMarkup Language (HTML), eXtensible Markup Language (XML), HTML5, and thelike.

Memory 304 may include Random Access Memory (RAM), Read-Only Memory(ROM), and/or other types of memory. Memory 304 illustrates an exampleof computer-readable storage media (devices) for storage of informationsuch as computer-readable instructions, data structures, program modulesor other data. Memory 304 stores a basic input/output system (BIOS) 308for controlling low-level operation of mobile device 300. The memoryalso stores an operating system 306 for controlling the operation ofmobile device 300. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX′, or a specialized mobile computer communication operating systemsuch as Windows Mobile™, or the Symbian® operating system. The operatingsystem may include, or interface with a Java virtual machine module thatenables control of hardware components and/or operating systemoperations via Java application programs.

Memory 304 further includes one or more data storage 310, which can beutilized by mobile device 300 to store, among other things, applications320 and/or other data. For example, data storage 310 may also beemployed to store information that describes various capabilities ofmobile device 300. The information may then be provided to anotherdevice based on any of a variety of events, including being sent as partof a header during a communication, sent upon request, or the like. Datastorage 310 may also be employed to store social networking informationincluding address books, buddy lists, aliases, user profile information,or the like. Data storage 310 may further include program code, data,algorithms, and the like, for use by a processor, such as processor 302to execute and perform actions. In one embodiment, at least some of datastorage 310 might also be stored on another component of mobile device300, including, but not limited to, non-transitory processor-readableremovable storage device 336, processor-readable stationary storagedevice 334, or even external to the mobile device. Data storage 310 mayinclude, for example, raw data 312, index data 314 and intermediatesummaries 316.

Applications 320 may include computer executable instructions which,when executed by mobile device 300, transmit, receive, and/or otherwiseprocess instructions and data. Applications 320 may include, forexample, Search application 322, and Indexing Application 324. Otherexamples of application programs include calendars, search programs,email client applications, IM applications, SMS applications, Voice OverInternet Protocol (VOIP) applications, contact managers, task managers,transcoders, database programs, word processing programs, securityapplications, spreadsheet programs, games, search programs, and soforth.

Illustrative Network Device

FIG. 4 shows one embodiment of network device 400 that may be includedin a system implementing the invention. Network device 400 may includemany more or less components than those shown in FIG. 4. However, thecomponents shown are sufficient to disclose an illustrative embodimentfor practicing the present invention. Network device 400 may represent,for example, one embodiment of at least one of network device 112, 114,or 120 of FIG. 1.

As shown in the figure, network device 400 includes a processor 402 incommunication with a memory 404 via a bus 428. Network device 400 alsoincludes a power supply 430, network interface 432, audio interface 456,display 450, keyboard 452, input/output interface 438,processor-readable stationary storage device 434, and processor-readableremovable storage device 436. Power supply 430 provides power to networkdevice 400.

Network interface 432 includes circuitry for coupling network device 400to one or more networks, and is constructed for use with one or morecommunication protocols and technologies including, but not limited to,protocols and technologies that implement any portion of the OpenSystems Interconnection model (OSI model), global system for mobilecommunication (GSM), code division multiple access (CDMA), time divisionmultiple access (TDMA), user datagram protocol (UDP), transmissioncontrol protocol/Internet protocol (TCP/IP), Short Message Service(SMS), Multimedia Messaging Service (MMS), general packet radio service(GPRS), WAP, ultra wide band (UWB), IEEE 802.16 WorldwideInteroperability for Microwave Access (WiMax), Session InitiationProtocol/Real-time Transport Protocol (SIP/RTP), or any of a variety ofother wired and wireless communication protocols. Network interface 432is sometimes known as a transceiver, transceiving device, or networkinterface card (NIC). Network device 400 may optionally communicate witha base station (not shown), or directly with another computing device.

Audio interface 456 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 456 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others and/or generate an audio acknowledgementfor some action. A microphone in audio interface 456 can also be usedfor input to or control of network device 400, for example, using voicerecognition.

Display 450 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computing device. Display 450 may be a handheld projector orpico projector capable of projecting an image on a wall or other object.

Network device 400 also may also comprise input/output interface 438 forcommunicating with external devices not shown in FIG. 4. Input/outputinterface 438 can utilize one or more wired or wireless communicationtechnologies, such as USB™, Firewire™, WiFi, WiMax, Thunderbolt™,Infrared, Bluetooth™, Zigbee™, serial port, parallel port, and the like.

Human interface components can be physically separate from networkdevice 400, allowing for remote input and/or output to network device400. For example, information routed as described here through humaninterface components such as display 450 or keyboard 452 can instead berouted through the network interface 432 to appropriate human interfacecomponents located elsewhere on the network. Human interface componentsinclude any component that allows the computer to take input from, orsend output to, a human user of a computer.

Memory 404 may include Random Access Memory (RAM), Read-Only Memory(ROM), and/or other types of memory. Memory 404 illustrates an exampleof computer-readable storage media (devices) for storage of informationsuch as computer-readable instructions, data structures, program modulesor other data. Memory 404 stores a basic input/output system (BIOS) 408for controlling low-level operation of network device 400. The memoryalso stores an operating system 406 for controlling the operation ofnetwork device 400. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX′, or a specialized operating system such as MicrosoftCorporation's Windows® operating system, or the Apple Corporation's IOs®operating system. The operating system may include, or interface with aJava virtual machine module that enables control of hardware componentsand/or operating system operations via Java application programs.

Memory 404 further includes one or more data storage 410, which can beutilized by network device 400 to store, among other things,applications 420 and/or other data. For example, data storage 410 mayalso be employed to store information that describes variouscapabilities of network device 400. The information may then be providedto another device based on any of a variety of events, including beingsent as part of a header during a communication, sent upon request, orthe like. Data storage 410 may also be employed to store socialnetworking information including address books, buddy lists, aliases,user profile information, or the like. Data stores 410 may furtherinclude program code, data, algorithms, and the like, for use by aprocessor, such as processor 402 to execute and perform actions. In oneembodiment, at least some of data store 410 might also be stored onanother component of network device 400, including, but not limited to,non-transitory media inside processor-readable removable storage device436, processor-readable stationary storage device 434, or any othercomputer-readable storage device within network device 400, or evenexternal to network device 400. Data storage 410 may include, forexample, raw data 412, indexed data 414, and intermediate summaries 416.

Applications 420 may include computer executable instructions which,when executed by network device 400, transmit, receive, and/or otherwiseprocess messages (e.g., SMS, Multimedia Messaging Service (MMS), InstantMessage (IM), email, and/or other messages), audio, video, and enabletelecommunication with another user of another mobile device. Otherexamples of application programs include calendars, search programs,email client applications, IM applications, SMS applications, Voice OverInternet Protocol (VOIP) applications, contact managers, task managers,transcoders, database programs, word processing programs, securityapplications, spreadsheet programs, games, search programs, and soforth. Applications 420 may include, for example, search application422, and indexing application 424.

General Operation

FIG. 5, illustrates a flowchart for general overview 500 of at least oneof the various embodiments for processing a search query for aparticular search report. Moving from a start block, the process stepsto block 502 where a data set is arranged in a plurality of partitionsbased at least in part on separate events over time.

Stepping to block 504, reducible search computations for the particularsearch report are performed for data in at least one of the partitions.A search computation is a logical sequence of actions performed on datato generate a result. A reducible search computation might generate anaverage, a variance, a root mean square, or the like that canpotentially be reused in one or more different search reports. Incontrast, a non-reducible search computation might instead requireaccessing most if not all of the data in each partition each time aparticular search report is requested. For example, the non-reduciblesearch computation might involve computing a statistic that is groupedby the value of a high cardinality field, such as a user ID or an IPaddress. Since this type of computation is unlikely to be significantlyreducible, the intermediate summaries would be on par with the size ofthe original data. In this case, the process would skip generating theintermediate summaries for this partition of the data or that particularsearch report. In at least one of the various embodiments, at least someof the reducible search computations may be generated by an applicationthat also enables indexing of the data into one or more partitions thatcorrespond to events over time.

Advancing to block 506, one or more intermediate summaries are generatedfor reducible data for the particular search report in each partition.Also, depending on the nature of the data in each partition, a partitionmay include one or more intermediate summaries, adhoc data, or acombination of both intermediate summaries and adhoc data. In at leastone of the various embodiments, at least some of the intermediatesummaries are generated by an application that also enables indexing ofthe data into one or more partitions that correspond to events overtime.

At block 508, in response to a search query for a particular searchreport, the intermediate summaries and the reducible search computationsthat correspond to the search report are aggregated for the data set.

Then, at block 510, a search result is generated for the particularsearch report based at least one of the aggregated intermediatesummaries, aggregated reducible search computations, and the processingof any remaining adhoc data in one or more partitions with the searchquery. Next, the process returns to performing other actions.

FIG. 6 illustrates overview flowchart 600 for processing intermediatesummaries and reducible search computations for a particular searchreport for data that is arranged in a plurality of partitions. Movingfrom a start block, the process advances to block 602 where the data setis arranged in a plurality of partitions that correspond to a time ofoccurrence for events. At decision block 604, a determination is made asto whether the structure of a particular requested search report isreducible. If negative, the logic loops back to block 602. However, ifthe determination at block 604 is affirmative, the process flows toblock 606 where reducible search computations are performed on data ineach partition for the particular search report. The values generated bythe reducible search computations are normalized for the data set.

At decision block 608, a determination is made as to whether reducibledata is present in a partition for a particular search report. If true,the process advances to block 610, where the process generatesintermediate summaries for reducible data in each partition for theparticular search report. In some cases, a very small portion of data ineach partition may be reducible, and/or the reduction in the data maynot be significant for a particular search report. Consequently, eventhough some intermediate summaries may be generated, the process maydetermine to not continue to do so if the advantage (reduction in datato be processed for a particular search report) is relatively small.

Moving to block 612, newly identified data for an event may be appendedto other data already arranged in a partition that corresponds to theevent. Also, if the determination at decision block 608 was false (noreducible data in at least one partition for a particular searchreport), the process would jump to block 612 and perform substantiallythe same actions as discussed above.

At decision block 614, if a period of time since the last processing ofthe partitioned data set has not yet elapsed, the process loops untiltrue and then loops back to block 602 where it performs substantiallythe same actions as discussed above.

FIG. 7 shows a flowchart of a general overview 700 for processing arequest for a particular search report. Moving from a start block, theprocess advances to block 702 where a query for a particular searchreport is received.

At block 704, the available intermediate summaries for the particularsearch report from one or more partitions are aggregated. Similarly, atblock 706, the available normalized reducible search computations forthe particular search report from one or more partitions are aggregated.

Moving to block 708, the search query for the particular search reportis performed on adhoc data that is non-reducible. Advancing to block710, a result for the particular search report is generated based atleast in part on the aggregated intermediate summaries, aggregatedreducible search computations, and the query performed on the adhocdata. Next, the process returns to performing other actions.

FIG. 8 illustrates a schematic diagram of data set 800 that includes aplurality of data partitions 802, 804, 806, 808, 810, and 812 thatcorrespond to events over time. As shown, partitions 802, 808 and 812include non-reducible adhoc data for a particular search report.Partition 804 includes 100 percent intermediate summaries for theparticular search report. Partitions 806 and 810 include partialintermediate summaries and partial adhoc data for the particular searchreport. Also, appended to partition 808 is newly identified data for theevent that corresponds to the partition.

FIG. 9 shows an exemplary table of raw data that includes columns fortime, name, size, and ip address on a server. Also, FIG. 10 illustratesan exemplary table of intermediate summaries that may be generated for aquery that needs the average and maximum of the size field over timesplit by name. In addition to time and name, the table includes thecount of the size field, the maximum of the size field, and the sum ofthe size field for each value of time and name. This may be because inat least one of the various embodiments a separate accounting of thecount and the sum may be maintained to compute an accurate overallaverage (arithmetic mean).

It will be understood that figures, and combinations of actions in theflowchart-like illustrations, can be implemented by computer programinstructions. These program instructions may be provided to a processorto produce a machine, such that the instructions executing on theprocessor create a means for implementing the actions specified in theflowchart blocks. The computer program instructions may be executed by aprocessor to cause a series of operational actions to be performed bythe processor to produce a computer implemented process for implementingthe actions specified in the flowchart block or blocks. These programinstructions may be stored on some type of machine readable storagemedia, such as processor readable non-transitive storage media, or thelike.

Furthermore, it will be understood that for at least one of the variousembodiments, various types of data may be received and processed asdescribed and claimed herein. And, at least one of the variousembodiments is not limited to processing machine data.

What is claimed as new and desired to be protected by Letters Patent ofthe United States is:
 1. A method, comprising: providing a datastorecomprising a plurality of time-stamped events, each event having aportion of raw data and a timestamp extracted from the portion of rawdata; receiving a query to produce a report to be automaticallygenerated and updated with new event data corresponding to eventsreceived during periodic intervals between reports; determining if eventdata responsive to the query is reducible, wherein reducible event datais used to update the report at the periodic intervals; responsive to adetermination that the event data responsive to the query is reducible,generating one or more intermediate summaries of the event dataresponsive to the query; and automatically generating the report at theperiodic intervals using the one or more intermediate summaries.
 2. Themethod of claim 1, wherein the generating comprises generating the oneor more intermediate summaries of the event data responsive to the queryfor distinct time periods in a time range specified by the query for thereport, wherein at least one of the distinct time periods occurs duringone of the periodic intervals between the reports.
 3. The method ofclaim 1, wherein automatically generating the report using the one ormore intermediate summaries comprises: aggregating the one or moreintermediate summaries; and generating the report from data in the oneor more intermediate summaries.
 4. The method of claim 1, whereinautomatically generating the report using the one or more intermediatesummaries further comprises: determining that additional event datarelevant to the report exists that was not summarized in the one or moreintermediate summaries; and generating the report using the one or moreintermediate summaries and the additional event data.
 5. The method ofclaim 1, wherein automatically generating the report using the one ormore intermediate summaries further comprises: determining thatadditional event data relevant to the report exists that was notsummarized in the one or more intermediate summaries; and generating thereport using the one or more intermediate summaries and the additionalevent data, wherein data contained in the one or more intermediatesummaries is reusable for future search queries and wherein theadditional event data is unique to the report.
 6. The method of claim 1,wherein automatically generating the report using the one or moreintermediate summaries further comprises: determining that additionalevent data relevant to the report exists that was not summarized in theone or more intermediate summaries, wherein the additional event datadoes not correspond to distinct time periods; and generating the reportusing the one or more intermediate summaries and the additional eventdata.
 7. The method of claim 1, further comprising: receiving a secondquery to automatically generate a second report; determining if the oneor more intermediate summaries can be reused to generate the secondreport; responsive to a determination that the one or more intermediatesummaries can be reused, aggregating the one or more intermediatesummaries of the event data; and automatically generating the secondreport using the aggregated intermediate summaries.
 8. The method ofclaim 1, wherein the one or more intermediate summaries are operable tobe generated by an application that indexes the raw data into one ormore partitions comprising the time-stamped events.
 9. The method ofclaim 1, wherein the data stored in the datastore is partitioned basedon time occurrence of the time-stamped events.
 10. A non-transitorycomputer-readable medium storing computer-executable instructions which,when executed by a processor, cause the processor to perform operationscomprising: providing a datastore comprising a plurality of time-stampedevents, each event having a portion of raw data and a timestampextracted from the portion of raw data; receiving a query to produce areport to be automatically generated and updated with new event datacorresponding to events received during periodic intervals betweenreports; determining if event data responsive to the query is reducible,wherein reducible event data is used to update the report at theperiodic intervals; responsive to a determination that the event dataresponsive to the query is reducible, generating one or moreintermediate summaries of the event data responsive to the query; andautomatically generating the report at the periodic intervals using theone or more intermediate summaries.
 11. The non-transitorycomputer-readable medium of claim 10, wherein the generating comprisesgenerating the one or more intermediate summaries of the event dataresponsive to the query for distinct time periods in a time rangespecified by the query for the report, wherein at least one of thedistinct time periods occurs during one of the periodic intervalsbetween the reports.
 12. The non-transitory computer-readable medium ofclaim 10, wherein automatically generating the report using the one ormore intermediate summaries comprises: aggregating the one or moreintermediate summaries; and generating the report from data in the oneor more intermediate summaries.
 13. The non-transitory computer-readablemedium of claim 10, wherein automatically generating the report usingthe one or more intermediate summaries further comprises: determiningthat additional event data relevant to the report exists that was notsummarized in the one or more intermediate summaries; and generating thereport using the one or more intermediate summaries and the additionalevent data.
 14. The non-transitory computer-readable medium of claim 10,wherein automatically generating the report using the one or moreintermediate summaries further comprises: determining that additionalevent data relevant to the report exists that was not summarized in theone or more intermediate summaries; and generating the report using theone or more intermediate summaries and the additional event data,wherein data contained in the one or more intermediate summaries isreusable for future search queries and wherein the additional event datais unique to the report.
 15. The non-transitory computer-readable mediumof claim 10, wherein automatically generating the report using the oneor more intermediate summaries further comprises: determining thatadditional event data relevant to the report exists that was notsummarized in the one or more intermediate summaries, wherein theadditional event data does not correspond to distinct time periods; andgenerating the report using the one or more intermediate summaries andthe additional event data.
 16. The non-transitory computer-readablemedium of claim 10, the operations further comprising: receiving asecond query to automatically generate a second report; determining ifthe one or more intermediate summaries can be reused to generate thesecond report; responsive to a determination that the one or moreintermediate summaries can be reused, aggregating the one or moreintermediate summaries of the event data; and automatically generatingthe second report using the aggregated intermediate summaries.
 17. Thenon-transitory computer-readable medium of claim 10, wherein the one ormore intermediate summaries are operable to be generated by anapplication that indexes the raw data into one or more partitionscomprising the time-stamped events.
 18. The non-transitorycomputer-readable medium of claim 10, wherein the data stored in thedatastore is partitioned based on time occurrence of the time-stampedevents.
 19. A system comprising: at least one memory storingcomputer-executable instructions; and at least one processor, whereinthe at least one processor is configured to access the at least onememory and to execute the computer-executable instructions to: provide adatastore comprising a plurality of time-stamped events, each eventhaving a portion of raw data and a timestamp extracted from the portionof raw data; receive a query to produce a report to be automaticallygenerated and updated with new event data corresponding to eventsreceived during periodic intervals between reports; determine if eventdata responsive to the query is reducible, wherein reducible event datais used to update the report at the periodic intervals; responsive to adetermination that the event data responsive to the query is reducible,generate one or more intermediate summaries of the event data responsiveto the query; and automatically generate the report at the periodicintervals using the one or more intermediate summaries.
 20. The systemof claim 19, wherein to automatically generate the report the at leastone processor is configured to generate the one or more intermediatesummaries of the event data responsive to the query for distinct timeperiods in a time range specified by the query for the report, whereinat least one of the distinct time periods occurs during one of theperiodic intervals between the reports.
 21. The method of claim 19,wherein to automatically generate the report using the one or moreintermediate summaries, the at least one processor is configured to:aggregate the one or more intermediate summaries; and generate thereport from data in the one or more intermediate summaries.
 22. Themethod of claim 19, wherein to automatically generate the report usingthe one or more intermediate summaries, the at least one processor isconfigured to: determine that additional event data relevant to thereport exists that was not summarized in the one or more intermediatesummaries; and generate the report using the one or more intermediatesummaries and the additional event data.
 23. The method of claim 19,wherein to automatically generate the report using the one or moreintermediate summaries, the at least one processor is configured to:determine that additional event data relevant to the report exists thatwas not summarized in the one or more intermediate summaries; andgenerate the report using the one or more intermediate summaries and theadditional event data, wherein data contained in the one or moreintermediate summaries is reusable for future search queries and whereinthe additional event data is unique to the report.
 24. The method ofclaim 19, wherein to automatically generate the report using the one ormore intermediate summaries, the at least one processor is configuredto: determine that additional event data relevant to the report existsthat was not summarized in the one or more intermediate summaries,wherein the additional event data does not correspond to distinct timeperiods; and generate the report using the one or more intermediatesummaries and the additional event data.
 25. The method of claim 19,wherein the at least one processor is further configured to: receive asecond query to automatically generate a second report; determine if theone or more intermediate summaries can be reused to generate the secondreport; responsive to a determination that the one or more intermediatesummaries can be reused, aggregate the one or more intermediatesummaries of the event data; and automatically generate the secondreport using the aggregated intermediate summaries.
 26. The method ofclaim 19, wherein the one or more intermediate summaries are operable tobe generated by an application that indexes the raw data into one ormore partitions comprising the time-stamped events.
 27. The method ofclaim 19, wherein the data stored in the datastore is partitioned basedon time occurrence of the time-stamped events.